A Cost-Shaping Linear Program for Average-Cost Approximate Dynamic Programming with Performance Guarantees

We introduce a new algorithm based on linear programming for optimization of average-cost Markov decision processes (MDPs). The algorithm approximates the differential cost function of a perturbed MDP via a linear combination of basis functions. We establish a bound on the performance of the resulting policy that scales gracefully with the number of states without imposing the strong Lyapunov condition required by its counterpart in [8]. We investigate implications of this result in the context of a queueing control problem.